Skip to content

SOLR-18060: CrossDC Consumer - add Prometheus metrics.#4044

Closed
sigram wants to merge 4 commits intoapache:mainfrom
sigram:jira/solr-18060
Closed

SOLR-18060: CrossDC Consumer - add Prometheus metrics.#4044
sigram wants to merge 4 commits intoapache:mainfrom
sigram:jira/solr-18060

Conversation

@sigram
Copy link
Contributor

@sigram sigram commented Jan 12, 2026

This PR adds /metrics/prometheus to the CrossDC Consumer, in addition to the existing Dropwizard JSON metrics endpoint (moved under /metrics/json).

Prometheus 1.4.3 was selected for this module for several reasons:

  • 1.1.0 is over 2 years old and there have been many important bug fixes since then.
  • Similarly, Java simpleclient is EOL since 1.0.0.
  • 1.1.0 doesn't have a built-in support for wrapping Dropwizard metrics. Importing just the relevant parts would create additional tech debt for Solr.

@mlbiscoc
Copy link
Contributor

mlbiscoc commented Jan 13, 2026 

# HELP crossdc_consumer_UPDATE_add_total Generated from Dropwizard metric import (metric=UPDATE.add, type=com.codahale.metrics.Counter)
    # TYPE crossdc_consumer_UPDATE_add_total counter
crossdc_consumer_UPDATE_add_total 1.0
    # HELP crossdc_consumer_UPDATE_handled_total Generated from Dropwizard metric import (metric=UPDATE.handled, type=com.codahale.metrics.Counter)
    # TYPE crossdc_consumer_UPDATE_handled_total counter
crossdc_consumer_UPDATE_handled_total 1.0
    # HELP crossdc_consumer_UPDATE_input_total Generated from Dropwizard metric import (metric=UPDATE.input, type=com.codahale.metrics.Counter)
    # TYPE crossdc_consumer_UPDATE_input_total counter
crossdc_consumer_UPDATE_input_total 1.0
    # HELP crossdc_consumer_UPDATE_outputLatency Generated from Dropwizard metric import (metric=UPDATE.outputLatency, type=com.codahale.metrics.Timer)
    # TYPE crossdc_consumer_UPDATE_outputLatency summary
crossdc_consumer_UPDATE_outputLatency{quantile="0.5"} 0.0068465
crossdc_consumer_UPDATE_outputLatency{quantile="0.75"} 0.0068465
crossdc_consumer_UPDATE_outputLatency{quantile="0.95"} 0.0068465
crossdc_consumer_UPDATE_outputLatency{quantile="0.98"} 0.0068465
crossdc_consumer_UPDATE_outputLatency{quantile="0.99"} 0.0068465
crossdc_consumer_UPDATE_outputLatency{quantile="0.999"} 0.0068465
crossdc_consumer_UPDATE_outputLatency_count 1
    # HELP crossdc_consumer_UPDATE_outputTime Generated from Dropwizard metric import (metric=UPDATE.outputTime, type=com.codahale.metrics.Timer)
    # TYPE crossdc_consumer_UPDATE_outputTime summary
crossdc_consumer_UPDATE_outputTime{quantile="0.5"} 0.020751709
crossdc_consumer_UPDATE_outputTime{quantile="0.75"} 0.020751709
crossdc_consumer_UPDATE_outputTime{quantile="0.95"} 0.020751709
crossdc_consumer_UPDATE_outputTime{quantile="0.98"} 0.020751709
crossdc_consumer_UPDATE_outputTime{quantile="0.99"} 0.020751709
crossdc_consumer_UPDATE_outputTime{quantile="0.999"} 0.020751709
crossdc_consumer_UPDATE_outputTime_count 1

I am really not a fan of the Dropwizard -> Prometheus bridges that exist because it hacks around prometheus labels which is its main feature and the metrics look a bit low quality in its name. It would also be nice to try and remove any uses of Dropwizard and this PR will tightly couple it and users to this if when we just moved to OTEL in core. Being that this CrossDC is in it's own module and there are not that many metrics, maybe its ok?

@sigram
Copy link
Contributor Author

sigram commented Jan 13, 2026

@mlbiscoc hmm, I see your point. I didn't want to make too many changes in one PR but maybe it's best to bite the bullet and simply port the Consumer over to Otel. I'll look into it.

@sigram
Copy link
Contributor Author

sigram commented Jan 19, 2026

See PR #4063 for alternative approach.

@sigram sigram closed this Jan 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@HoustonPutman HoustonPutman Awaiting requested review from HoustonPutman

@mlbiscoc mlbiscoc Awaiting requested review from mlbiscoc

Assignees

No one assigned

Labels

admin-ui cat:api client:solrj dependencies Dependency upgrades documentation Improvements or additions to documentation module:clustering module:extraction module:gcs-repository module:jwt-auth module:langid module:ltr module:opentelemetry module:s3-repository module:scripting module:sql test-framework tests tool:build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@sigram @mlbiscoc